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1 Introduction 


Stackelberg security games played between a defender (leader) and an attacker (follower) have been 
widely studied in the past few years Eunmnmg. Most models, in particular, including all the 
deployed security systems in m, assume that the attacker is not able to observe (even partially) 
the defender’s instantiated pure strategy (i.e., which targets are being protected), thus he makes 
decisions based only on his knowledge of the defender’s mixed strategy. This fails to capture the 
attacker’s real-time surveillance, by which he may partially observe the deployed pure strategy. 
For example, the attacker may observe the protection status of a certain target while approaching 
for an attack; or in some security domains information regarding the protection status of certain 
targets may leak to the attacker due to real-time surveillance or even an insider threat; further, 
well-prepared attackers may approach certain adversarially chosen target to collect information 
before committing an attack. 

Unfortunately, this problem- an issue we refer to as information leakage - has not received 
much attention in Stackelberg security games. In the literature of patrolling games, attackers’ 
real-time surveillance is indeed considered 12 DJ El H 0 ET]. However, all these papers study 
settings of patrols carried out over space and time, i.e., the defender follows a schedule of visits to 
multiple targets over time. In addition, they assume that it takes time for the attacker to execute 
an attack, during which the defender can interrupt the attacker by visiting the attacked target. 
Therefore, even if the attacker can fully observe the current position of the defender (in essence, 
status of all targets), he may not have enough time to complete an attack on a target before being 
interrupted by the defender. The main challenge there is to create patrolling schedules with the 
smallest possible time between any two target visits. In contrast, we consider information leakage 
in standard security game models, where the attack is instantaneous and cannot be interrupted 
by the defender’s resource re-allocation. Furthermore, as may be more realistic in our settings, 
we assume that information is leaked from a limited number of targets. As a result, our setting 
necessitates novel models and techniques. We also provide efficient algorithms with complexity 
analysis. 

This paper considers the design of optimal defender strategy in the presence of partial informa¬ 
tion leakage. Considering that real-time surveillance is costly in practice, we explicitly assume that 
information leaks from only one target, though our model and algorithms can be generalized. We 
start from the basic security game model where the defender allocates k resources to protect n tar¬ 
gets without any scheduling constraint. Such models have applications in real security systems like 
ARMOR for LAX airport and GUARDS for airports in general [15). We first show via a concrete 
example in Section [2] how ignoring information leakage can lead to significant utility loss. This 
motivates our design of optimal defending strategy given the possibility of information leakage. 
We start with a linear program formulation. However, surprisingly, we show that it is difficult to 
solve the LP even for this basic case, whereas the optimal mixed strategy without leakage can be 
computed easily. In particular, we show that the defender oracle, a key subproblem used in the col¬ 
umn generation technique employed for most security games, is NP-hard. This shows the intrinsic 
difficulty of handling information leakage. We then approach the problem from three directions: 
efficient algorithms for special cases, approximation algorithms and heuristic algorithms for sam¬ 
pling that improves upon the status quo. Our experiments support our hypothesis that ignoring 
information leakage can result in significant loss of utility for the defender, and demonstrates the 
value of our algorithms. 


2 


2 Model of Information Leakage 


Consider a standard zero-sum Stackelberg security game with a defender and an attacker. The 
defender allocates k security resources to protect n targets, which are denoted by the set [n] = 
{1, 2,n}. In this paper we consider the case where the security resources do not have scheduling 
constraints. That is, the defender’s pure strategy is to protect any subset of [n] of size at most k. For 
any i E [n], let vy be the reward [c t be the cost] of the defender when the attacked target i is protected 
[unprotected]. We consider zero-sum games, therefore the attacker’s utility is the negation of the 
defender’s utility. Let s denote a pure strategy and S be the set of all possible pure strategies. 
With some abuse of notation, we sometimes regard s as a subset of [ra] denoting the protected 
targets; and sometimes view it as an n-dimensional 0 — 1 vector with k l’s specifying the protected 
targets. The intended interpretation should be clear from context. The support of a mixed strategy 
is defined to be the set of pure strategies with non-zero probabilities. Without information leakage, 
the problem of computing the defender’s optimal mixed strategy can be compactly formulated as 
linear program |l]) with each variable ay as the marginal probability of covering target i. The 
resulting marginal vector if is a convex combination of the indicator vectors of pure strategies, and 
a mixed strategy with small support can be efficiently sampled, e.g., by Comb Sampling [16]. 

maximize u 

subject to u < r{Xi + cy( 1 — ay), for i E [n] 

Ei 6 [n] ^ k 

0 < ay < 1, for i E [n] 

Building on this basic security game, our model goes one step further and considers the possi¬ 
bility that the protection status of one target leaks to the attacker. Here, by “protection status” we 
mean whether this target is protected or not in an instantiation of the mixed strategy. We consider 
two related models of information leakage: 

1. PRobabilistic Information leakage (PRIL): with probability Pi(> 0) a single target i leaks 
information; and with probability po = 1 — X^=i Pi no targets leak information. So we have 
P = (PO)Pi) • • •, Pn ) £ A n+ i where A n+ i is the (n + l)-dimensional simplex. In practice, p is 
usually given by domain experts and may be determined by the nature or property of targets. 

2. ADversarial Information Leakage (ADIL): with probability 1 — pq, one adversarially chosen 
target leaks information. This model captures the case where the attacker will strategically 
choose a target for surveillance and with certain probability he succeeds in observing the 
protection status of the surveyed target. 

Given either model - PRIL with any p E A n +i or ADIL - we are interested in computing the 
optimal defender patrolling strategy. The first question to ask is: why does the issue of information 
leakage matter and how does it affect the computation of the optimal defender strategy? To answer 
this question we employ a concrete example. 

Consider a zero-sum security game with 4 targets and 2 resources. The profiles of reward r, [cost 
cy] is r = (1,1, 2, 2) [c = (—2, —2, —1, —1)], where the coordinates are indexed by target ids. If there 
is no information leakage, it is easy to see that the optimal marginal coverage is x = 

The attacker will attack an arbitrary target, resulting in a defender utility of 0. Now, let us consider 
a simple case of information leakage. Assume the attacker observes whether target 1 is protected 
or not in any instantiation of the mixed strategy, i.e., p\ = 1. As we will argue, how the marginal 
probability x is implemented would matter now. One way to implement x is to protect target {1, 2} 


( 1 ) 
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with probability | and protect {3,4} with probability /. However, this implementation is “fragile” 
in the presence of the above information leakage. In particular, if the attacker observes that target 
1 is protected (which occurs with probability |), he infers that the defender is protecting target 
{1, 2} and will attack 3 or 4, resulting in a defender utility of —1; if target 1 is not protected, the 
attacker will just attack, resulting in a defender utility of —2. Therefore, the defender gets expected 
utility — 

Now consider another way to implement the same marginal x by the following mixed strategy: 


{1,2} 

{1,3} 

{1,4} 

{2,3} 

{2,4} 

{3,4} 

10/27 

4/27 

4/27 

4/27 

4/27 

1/27 


If the attacker observes that target 1 is protected (which occurs with probability |), then he 

in 

infers that target 2 is protected with probability 1() 2 / 4 = |, and target 3,4 are both protected 

27 ' 27 ' 27 

with probability Some calculation shows that the attacker will have the same utility ^ on target 
2, 3,4 and thus will choose an arbitrary one to attack, resulting in a defender utility of — On the 
other hand, if target 1 is observed to be unprotected, the defender gets utility —2. In expectation, 
the defender gets utility | x (— |) + 1 x (—2) = — 

As seen above, though implementing the same marginals, the latter mixed strategy achieves 
better defender utility than the former one in the presence of information leakage. However, is it 
optimal? It turns out that the following mixed strategy achieves an even better defender utility of 
— which can be proved to be optimal: protect {1,2} with probability |, {1,3} with probability 
| and {1,4} with probability |. 

This example shows that compact representation by marginal coverage probabilities is not suf¬ 
ficient for computing the optimal defending strategy assuming information leakage. This naturally 
raises new computational challenges: how can we formulate the defender’s optimization problem 
and compute the optimal solution? Is there still a compact formulation or is it necessary to enu¬ 
merate all the exponentially many pure strategies? What is the computational complexity of this 
problem? We answer these questions in the next section. 

3 Computing Optimal Defender Strategy 

We will focus on the derivation of the PRIL model. The formulation for the ADIL model is provided 
at the end of this section since it admits a similar derivation. Fixing the defender’s mixed strategy, 
let ti (-■ ti ) denote the event that target i is protected ( unprotected ). For the PRIL model, the 
defender’s utility equals 

DefU = PqU + YA=lPi(. u i + v i) 

where u = rnirij [rj Pr (tj) + Cj Pr(-ifj)] is the defender’s utility when there is no information leak¬ 
age; and 

Ui = Pr (ti) X min, [rj Pr(t,|L) + Cj Pr(-.^|L)] 

= rniiij [rj Pr(tj,ti) + Cj Pr(-.^-,L)] 

is the defender’s utility when target i leaks out its protection status as L (i.e., protected) multiplied 
by probability Pr (ti). Similarly 

Vi = min, [rj Pr (tj, ->L) + Cj Pr (—-tj, —■**)] 

is the defender’s expected utility multiplied by probability Pr(-itj) when target i leaks status —it* 
(i.e., unprotected) 
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Define variables x^ = Pr(fj, tj) (setting x„ = Pr(f,)). Using the fact that Pr(fj, -i tj) = xu — Xij 
and Pr(-i£j, -i tj) = 1 — xu — Xjj + Xij, we obtain the following linear program which computes the 
defender’s optimal patrolling strategy: 


maximize p 0 u + Ya=i Pi( u i + v i) 
subject to u < VjXjj + Cj( 1 — Xjj), 

Or U VjXij “1“ Cj{xii X ij j, 

Vi < Tj(Xjj - Xij) + Cj(l - Xu ~ Xjj + Xij), 
Xij — ^2s:i,jes 9s ’ 

Sses = 1 
9s > 0 , 


for j G [n]. 
for i, j G [n]. 
for z, j G [n]. 
for z, j G [n]. 

for s G 5. 


( 2 ) 


where u,Ui,Vi,Xij,6 s are variables; s denotes a pure strategy and the sum condition “s :i,j G s” 
means summing over all the pure strategies that protect both targets z and j (or i if i = j); 9 S 
denotes the probability of choosing strategy s. 

Unfortunately, LP ([2]) suffers from an exponential explosion of variables, specifically, 9 S . From 
the sake of computational efficiency, one natural idea is to find a compact representation of the 
defender’s mixed strategy. As suggested by LP ([2]), the variables x^, indicating the probability that 
targets z, j are both protected, are sufficient to describe the defender’s objective and the attacker’s 
incentive constraints. 

Let us call variables x^ the pair-wise marginals and think of them as a matrix X G M nxn , i.e., 
the z’th row and j’th column of X is Xij (not to be confused with the marginals x). We say X is 
feasible if there exists a mixed strategy, i.e., a distribution over pure strategies, that achieves the 
pair-wise marginals X. Clearly, not all X G M nxn are feasible. Let V(n,k) G M nxn be the set of 
all feasible X. The following lemma shows a structural property of V(n, k). 

Lemma 1. V(n,k) is a polytope and any X G V(n,k) is a symmetric positive semi-definite 
(PSD) matrix. 

Proof. Notice that X is feasible if and only there exists 9 S for any pure strategy s such that the 
following linear constraints hold: 


x ij = T,s:i,jes 9 si for i,j G [n\. 

YjsGS = 1 ( 3 ) 

9 S > 0, for sGS. 


These constraints define a polytope for variables (X,6), therefore its projection to the lower 
dimension X, which is precisely V(n, k), is also a polytope. 

To prove X G?(n, k) is PSD, we first observe that any vertex of V{n, k), characterizing a pure 
strategy, is PSD. In fact, let s G {0, l} n be any pure strategy, then the pair-wise marginal w.r.t. s 
is X s = ss T , which is PSD. Therefore, any X G V, which is a convex combination of its vertices, is 
also PSD. □ 


maximize p 0 u + Y17=i + v i) 

subject to u < rjXjj + c 3 {\ — Xjj), for j G [n]. 

Ui < VjXij + Cj(xu - Xij), for i,j G [n\. (4) 

Vi < Vjfxjj - Xij) + Cj(l - Xu - Xjj + Xij), for i,j G [n]. 

X €V(n,k) 
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With Lemma [TJ we may re-write LP ([ 2 ]) compactly as LP Q with variables u, Ui, Vi and X. 
Therefore, we would be able to compute the optimal strategy efficiently in polynomial time if the 
constraints determining the polytope V(n, k ) were only polynomially many - recall that this is the 
approach we took with LP ([!]) in the case of no information leakage. However, perhaps surprisingly, 
the problem turns out to be much harder in the presence of leakage. 

Lemma 2. Optimizing over V(n,k) is NP-hard. 

Proof. We prove by reduction from the densest fc-subgraph problem. Given any graph instance 
G = (V, E), let A be the adjacency matrix of G. Consider the following linear program: 

maximize £^ e[n] A ijXij 
subject to I £ v(n,k). 

This linear program must have a vertex optimal solution X* which satisfies X* = ss T for some 
pure strategy s G {0, l} n . Therefore, the linear objective satisfies 

AijXij = tr(AX*) = tr(A x ss T ) = tr(s T As ) = s T As. 

i,j€[n] 

Notice that s T As/2k equals the density of a subgraph of G with k nodes indicated by s. Since X* 
is the optimal solution to LP (|5|, it also maximizes the density s T As/2k over all subgraphs with k 
nodes. In other words, the ability of optimizing LP ([ 5 ]) implies the ability of computing the densest 
/^-subgraph, which is NP-hard. Therefore, optimizing over V{n, k ) is NP-hard. □ 

Lemma [2] suggests that there is no hope of finding polynomially many linear constraints which 
determine V(n, k ) or, more generally, an efficient separation oracle for V(n, k ), assuming P ^ NP. 
In fact, V(n, k ) is closely related to a fundamental geometric object, known as the correlation poly¬ 
tope, which has applications in quantum mechanics, statistics, machine learning and combinatorial 
problems. We show a connection between V(n, k ) and the correlation polytope in Appendix B. For 
further information, we refer the reader to H3i- 

Another approach for computing the optimal defender strategy is to use the technique of column 
generation, which is a master/slave decomposition of an optimization problem. The essential part 
of this approach is the slave problem, which is also called the “defender best response oracle” or 
“defender oracle” for short [6]. We defer the derivation of the defender oracle to Appendix A, while 
only mention that a similar reduction as in the proof of Lemma [2] also implies the follows. 

Lemma 3. The defender oracle is NP-hard. 

By now, we have shown the evidence of the difficulty of solving LP ([ 2 ]) using either marginals 
or the technique of column generation. For the ADIL model, a similar argument yields that the 


following LP formulation computes the optimal defender strategy. It 

is easy to check that it shares 

the same marginals and defender oracle as the PRIL model. 



maximize Pou+ (1 — po)w 

subject to u < rjXjj + Cj( 1 — Xjj ), 

for j G [n]. 


Oj T r j Xij T Cj{xa Xij ), 

for i,j G [n]. 

(6) 

Vi < rj(xjj - Xij) + Cj(l - Xu - Xjj + Xij) 

, for i,j G [rtj. 

W <Ui + Vi, 

X €V(n,k) 

for * G [n]. 



where variable w is the defender’s expected utility when an adversarially chosen target is observed 
by the attacker. 
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3.1 Leakage from Small Support of Targets 

Despite the hardness results for the general case, we show that the defender oracle admits a poly¬ 
nomial time algorithm if information only leaks from a small subset of targets; we call this set the 
leakage support. By re-ordering the targets, we may assume without loss of generality that only the 
first m targets, denoted by set [m], could possibly leak information in both the PRIL and ADIL 
model. For the PRIL model, this means Pi = 0 for any i > m and for the ADIL model, this means 
the attacker only chooses a target in [m] for surveillance. 

Why does this make the problem tractable? Intuitively the reason is as follows: when infor¬ 
mation leaks from a small set of targets, we only need to consider the correlations between these 
leaking targets and others, which is a much smaller set of variables than in LP ([ 2 ]) or ©>• Restricted 
to a leakage support of size m, the defender oracle is the following problem (See Appendix A for 
the derivation). Let A be a symmetric matrix of the following block form 


A 


A m m A mm i 

■Am'm -T m'm ' 


(7) 


where m! = n — m; A r 


E 


for any integers m, m! is a sub-matrix and, crucially, A m i m i is 


a diagonal matrix. Given A of form 0. find a pure strategy s such that s T As is maximized. That 
is, the defender oracle identifies the siz e-k principle submatrix with maximum entry sum for any A 
of form 0. Note that m = n in general case. 

Before detailing the algorithm, we first describe some notation. Let A[i, :] be the Pth row 
of matrix A and diag(A) be the vector consisting of the diagonal entries of A. For any subset 
C\ , C 2 of [n], let Ac 1 ,c 2 be the submatrix of A consisting of rows in C\ and columns in C 2 , and 
sum(Ac 1 fi 2 ) = YlieCi jeC 2 ^ij be the entry sum of Ac 1: c 2 - The following lemma shows that 
Algorithm [l] solves the defender oracle. Our main insight is that for a pure strategy s to be optimal, 
once the set C = s 0 [to] is decided, its complement C = s\C can be explicitly identified, therefore 
we can simply brute-force search to find the best C C [m\. Lemma [I] provides the algorithm 
guarantee, which then yields the polynomial solvability for the case of small m (Theorem [l]) . 

Lemma 4. Let m be the size of the leakage support. Algorithm^ solves the defender oracle and 
runs in poly(n , k, 2 m ) time. In particular, the defender oracle admits a poly(n , k) time algorithm 
if m is a constant. 

Proof. First, it is easy to see that Algorithm 1 runs in poly(2 m ,n,k) time since the for-loop is 
executed at most 2 m times. We show that it solves the defender oracle problem. 

Let s denote the indices of the principle submatrix of A with maximum entry sum. Notice that 
s can also be viewed as a pure strategy. Let C = s n [m] and C = s\ C. We claim that, given 
C, C must be the set of indices of the largest k — \C\ values from the set {v m +i, ...,v n }, where v 
is defined as v = 2 Yliec : ] + diag(A). In other words, if we know C, the set C can be easily 
identified. To prove the claim, we re-write the sum(A StS ) as follows: 


sum(A S!S ) 





sum(A c ,c) 

+ 

2 sum(A C p) 

+ 

surn(Ac % c) 

sum(A c ,c ) 

+ 

2 sum(A C Q) 

+ 

sum(diag(A-Q ^)) 

sum(A c ,c ) 

+ 

sum(2 A. 

i,C 

+ diag(A(j-g)) 



i£C 



sum(A c ,c ) 

+ 

surn{v c ) 



vale 
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where v = 2£T gC ,A[i,:] + diag(A) and v-^j is the sub-vector of v with indices in C. Given C , 
sum(Ac ) c ) is fixed, therefore C must be the set of indices of the largest k — \C\ elements from 
i, ...,v n }. Algorithm 1 then loop over all the possible C C [m] (2 m many ) and identifies the 
optimal one, i.e., the one achieving the maximum vale■ □ 


Algorithm 1 Defender Oracle 
Input: matrix A of form Q. 

Output: a pure strategy s. 

1: for all C C [m] constrained by \C\ < k do 
2 : v = 2]T ieC ,A[v] +diag(A)-, 

3: Choose the largest k — \C\ values from the set {u m +i,..., v n }, and denote the set of their 

indices as C; 

4: Set vale = sum(Ac t c ) + sum{v c )\ 

5: end for 

6: return the pure strategy s = C U C with maximum vale- 


Theorem 1. (Polynomial Solvability ) There is an efficient poly(n,k) time algorithm which 
computes the optimal defender strategy in the PRIL and ADIL model, if the size of the leakage 
support m is a constant. 


3.2 An Approximation Algorithm 

We now consider approximation algorithms. Recall that information leakage is due to the correla¬ 
tion between targets, thus one natural way to minimize leakage is to allocate each resource indepen¬ 
dently with certain distributions. Naturally, the normalized marginal x* jk becomes a choice, where 
x* is the solution to LP Q. To avoid the waste of using multiple resources to protect the same 
target, we sample without replacement. Formally, the independent sampling without replacement 
algorithm proceeds as follows: 1. compute the optimal solution x* of LP (JTJ) ; 2. independently 
sample k elements from [n] without replacement using distribution x*/k. 

Zero-sum games exhibit negative utilities, therefore an approximation ratio in terms of utility is 
not meaningful. To analyze the performance of this algorithm we shift all the payoffs by a constant, 
— rnirij a , and get an equivalent constant-sum game with all non-negative payoffs. Theorem[2]shows 
that this algorithm is “almost” a (1 — ^—approximation to the optimal solution in the PRIL model, 
assuming information leaks out from any target i with equal probability pi = 1 ~ Po . We note that 
proving a general approximation ratio for any p G A n+ i turns out to be very challenging, intuitively 
because the optimal strategy adjusts according to different p while the sampling algorithm does 
not depend on p. However, experiments empirically show that the ratio does not vary much for 
different p on average (see Section [5]). 

Theorem 2. Assume each target leaks information with equal probability pi = 1 ~ Po . Letci > 0 
be the shifted cost and Ui n d ep Sampie be the defender utility achieved by independent sampling without 
replacement. Then we have: 


Uu 


indepSample — ] 

Ci is an additive loss to Opt(Lp[2]) ; which is usually small in security games. 


(^-l)\Opt(LP^-(l-p 0 ) 


ELF. 

n 


where (1 — po) 













Proof of Theorem [2] 

Let Y = Y(x) E M nxn be a function of any x E M n , where yij is the probability that target i,j are 
both protected using independent sampling without replacement. We first prove Lemma [5j which 
provides a lower bound regarding how good the pair-wise marginals in Y approximate the given 
marginals x. The difficulty of proving Lemma [5] lies at that Y does not have a close form in terms 
of x if we sample without replacement. Our proof is based on a coupling argument by relating the 
algorithm to independent sampling with replacement. 

Lemma 5. Given x, Y = Y(x) satisfies the following (in)equalities: 


Y ya = k: > 

i£[n] 


Vii > (1 - )xi, Vi E [n]; 

e 


Vij > ^ 

yu ~ y k - 1 


) x jj Vi, 7^ j. 


( 8 ) 

(9) 

( 10 ) 


Proof. The first equation is easy to see, since each sampled pure strategy has k different targets 
due to sampling without replacement. To prove the other two inequalities, we instead consider 
independent sampling with replacement. Similarly, define function Z = Z{x) E R nxn to a func¬ 
tion of x, where Z{ 3 is the probability that target i, j are protected together when sampling with 
replacement. Contrary to Y, Z has succinct close forms, therefore we can lower bound entries in 
Z. We first consider za. 


zu = 1 - (1 -Xi/k) k 

> 1 - e~ Xi 

> (1 - ~)Xi. 

e 


where we used the fact (1 — e)* < e 1 for any e E (0,1). Now we lower bound Zij/zn as follows. 


Zii 


1 - (1 - Xi/k) k 




1 - (1 - Xi/k) k 


l_ (l_a)‘__ 

1 k’ 1 - (1 - Xi/k) k 


x±\k 
k > 


iv rv Jb'i 




o~Xi 


k — Xi 


) fc ] 


(ii) 


where all the equations just follow the arithmetic, while the inequality uses the fact that (1 — 
~^) k < e~ Xj and — is a decreasing function of x E (0,1). We now upper-bound the term 
(1 — p) k — (1 — Rzj) k us i n g the formula a k — b k = (a — b) as follows 
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(1 - - (1 - J^r) k 

rv rv Ju i 

1 t=0 


k(k — Xi) 
XiXj 

- k- 1 


\fc-i-t 

k — Xi 


Plugging in the above upper bound back to Inequality El we thus have 




(D rft . rp . 

C- lAj^iAy J 


1, 


— e 

Xi 


Jfc- 1 

Xi 


> (1 - )Xj — Til 

e ' 2 e Xi — lk — 1 

, 1 . Xi 

,*- 2 1, 


where the last inequality is due to the fact that /(x) = -^fzi is a decreasing function for x E (0, 1) 
and is upper bounded by lim x ^o = 1- 

Therefore, we have ^ > (|5j — ^)xj. We then conclude our proofs by claiming that y n > 
and yij/yu > z^/zu- 

To prove our claim, we use a coupling argument. Consider the following two stochastic process 
(StoP): 

1. StoP 1 : at time t independently sample a random value it (E [n]) with probability Xi t /k for 
any t = 1,2,... until precisely k different elements from [n] show up. 

2. StoP 2 : at time t independently sample a random value q (e [n]) with probability Xi t /k for 
t = 1,2 

Let C 1 [C 2 ] denote all the possible random sequences generated by StoP 1 [StoP 2 ], and Cf [Cf] 
denote the subset of C 1 [C 2 ], which consists of all the sequences including at least one i. For any 
e E Cf, let C e be the subset of sequences in C ] , whose hrst k element is precisely e. Notice that 
any sequence in C 1 has at least length of k while any sequence in C 2 has precisely k elements. 
Furthermore, C e C Cf and C e n C e ' = 0 for any e, e' E Cf and efd. 

Now, think of each sequence as a probabilistic event generated by the stochastic process. Notice 
that P(e; StoP 2 ) = P(C e ; StoP 1 ) due to the independence of the sampling procedure, therefore, 
we have 


P(Cf] StoP 2 ) = 


< 


J2 ncstoP 2 ) 

e£Cf 

J2 PiCe-StoP 1 ) 

eeCf 

P(Cf; StoP 1 ) 
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However, P(C}\StoP l ) = yu and P(Cf\StoP 2 ) = za- This proves ya > za. 

Notice that yij/yu > Zij/zu is equivalent to P(e G Cj\e G Cf]StoP 2 ) > P(e G Cj\e G 

StoP 1 ). To prove this inequality, we claim that it is without loss of generality to assume 
the first sample is i in both processes. This is because, if the first i shows up at the f’th sample, 
moving i to the first position would not change the probability of the sequence due to independence 
between each sampling step. Conditioned on i is sampled first, a similar argument as above shows 
that the probability of Stochastic process StoP 1 generating j is at least the probability of stochastic 
process StoP 2 generating j. □ 

Let x* be the optimal solution to LP ([!]) and U* be the corresponding objective value - the 
defender optimal utility with no leakage. To prove Theorem 2, we start from comparing OPT(LP [2]) 
with U*. From the objective of LP ([2]), we know that u < U *, Ui < U* since U* is the best possibly 
utility using k resources, and Vi < Ci since if target i is uncovered, the defender gets utility at most 
Cj. Therefore, we have 


OPT(LP^ 


-I 

< Po u* + ^^ 5>*^* + (i-4)Si) 

n zJ 

i —1 

^ l . , 1“ P0\TT* , 1“ PO V^_ 

< (Po + k - )U H- } Ci 

n n 


( 12 ) 


where we used the equation X)ie[n] x *a = k- now examine U m( [ e pSample- A simple argument 
yields that, if Cj > 0 for all i and each target i is covered by probability at least ax* for any i. then 
the defender utility is at least aU*. Therefore, by Lemma [5] we have 


TT . ^ „ f-, ^\TT* l 1* 

UindepSample ^ Po(l )U / J Vii ( 7 )U 

6 Tl. K 1 e 

i=l 

. f k-2 1 1-po 


(13) 


where we used the fact that X)ie[n] tlti = k- Comparing Inequalities 
UindepSampie > (fel? ~ ^)[OPT(LP2) - ^=i Ci]- This concludes our 


(12) and (13), we have 
proof of Theorem j2j 


4 Sampling Algorithms 

From Caratheodory’s theorem we know that, given any marginal coverage x, there are many differ¬ 
ent mixed strategies achieving the same marginal x (e.g., see examples in Section [2]). Another way 
to handle information leakage is to generate the optimal marginal coverage x *, computed by LP ([I]) , 
with low correlation between targets. Such a “good” mixed strategy, e.g., the mixed strategy with 
maximum entropy, is usually supported on a pure strategy set of exponential size. In this section, 
we propose two sampling algorithms, which efficiently generate a mixed strategy with exponentially 
large support and are guaranteed to achieve any given marginal x. 


4.1 Max-Entropy Sampling 

Perhaps the most natural choice to achieve low correlation is the distribution with maximum entropy 
restricted to achieving the marginal x, which can be formulated as the solution of Convex Program 
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(CP) (14). However, naive approaches for CP (14) require exponential running time since there 


are 0( 2 n ) variables. Interestingly, it turns out that this be resolved. 

maximize — 0 s ln(0 s ) 

subject to £ s:igs 0 s = x h for i E [n]. 

6 S > 0, for s E S. 


(14) 


where variable 0 S is the probability of taking pure strategy s. 

Theorem 3. There is an efficient algorithm which runs in poly(n, k) time and outputs a pure 
strategy s with probability 0* for any pure strategy s E 5, where 0* is the optimal solution to Convex 
Program 0 (within machine precisioi £j]). 

The proof of Theorem [3] relies on Lemmas [6] and [7} Lemma [6] presents a compact representation 
of 0* based on the KKT conditions of CP (|14h and its dual - the unconstrained Convex Program (15): 


minimize 


} 0 ) = £S=i A®* + ln (£ sS 5 e p ')> 


(15) 


where variables /3 E M n and e “ 3 = Hj gs e ■ 1 * . We notice that the dual program (15) as well as 
the characterization of 0* in Lemma [b] are not new (e.g., see [ 14j ). and we state it for completeness. 


Our contribution lies at proving that CP (15) can be computed efficiently in poly(n, k ) time in our 
security game setting despite the summation £ se5 e _/3s of 0(2 k ) terms. 

Lemma 6. Let (5* E M n be the optimal solution to CP © and set on = e ^ for any i E [n], 
then the optimal solution of CP (14) satisfies 


o: = 


Oio 


£ 


seS 


CX.Q 


(16) 


where a s = nj gs O!j for any pure strategy s E S. 

Furthermore, (3* can be computed in poly(n , k) time. 


Proof. As proved in [14] , the a 1 above is precisely e where (3* is the optimal solution to CP (14) 


We show that (3* can be computed in poly(n,k ) time. Notice that CP (15) has n variables but an 
expression of exponentially many terms, specifically, £ sg 5 e -/3s . The essential difficulty of comput¬ 
ing /(/?) lies at computing the sum ^) sgS e^ s , since the other parts can be explicitly calculated in 
polynomial time. Fortunately the sum £ sg 5 exhibits some combinatorial structure,and com¬ 
binatorial algorithms could be employed for computation. In particular, we show that a dynamic 
program computes the sum ^) sgS e' ;3s in poly(n,k ) time. The algorithm for computing V/(/3) 
can be designed in a similar fashion, and hence left to the reader. Since a convex program can be 
solved efficiently in machine precision given the access to its function value and derivatives, we then 
conclude our proof by describing the following dynamic program to compute £ sg s e~^ s , given any 

ffi 

Notice that the set of all pure strategies consists of all the subsets of [n] of cardinality k. Let 
i = e~^ i and a s = n ,; gs o;j. We then build the following DP table T(i,j ) = £ s . sC m M =i a s , which 


a. 


sums over all the subsets of [j] of cardinality i. Our goal is to compute T(k,n ) = J2seS e ■ We 


1 Computers cannot solve general convex programs exactly due to possible irrational solutions. Therefore, our 

algorithm is optimal within machine precision, and we simply call it ’’solved”. 
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first initialize T(0, j) = 1 and T(j,j) = for any j. Then using the following update rule, we 

can build the DP table and compute T(fc, n) in poly(k,n ) time. 

T(i,j) = T(i,j - 1) + atjT(i - 1, j - 1). 

□ 


Our next lemma considers how to efficiently sample a pure strategy s from an exponentially 
large support with probability 9* represented by Equation (16). The algorithm, as detailed in 
Algorithm [2j simply goes through each target and adds it to the pure strategy with a specifically 
designed probability until exactly k targets are added. 


Algorithm 2 Max-Entropy Sampling 
Input: : a G [0,oo) n , k. 

Output: : a pure strategy s with |s| = k. 

1: Initialize: s = 0; the DP table T(0, j) = 1 and T(j,j) = II { =1 cti for any j G [n], 

2: Compute T(i,j) = X^-sC[j] | s | =i a s for any i, j satisfying i < k, j < n and 1 < i < j, using the 
following update rule 


T(i,j) = T(i,j - 1) +ajT(i- l,j - 1). 


3: Set i = k, j = n; 

4: while i > 0 do 

5: Sampling: independently add j to s with probability 

_ QLjT[i - 1, j - 1) 

6: if j added to s then 

7: i = i — 1; 

8 : end if 

9 : j = j ~ 1 ; 

10: end while 
11: return s. 


Lemma 7. Given any input a G [0,oo) n , Algorithm ^ runs in poly(k,n ) time and correctly 
samples a pure strategy s with probability 9 S = ^ —, where a s = IIig s aj. 

2^s6S Qs 

Proof. It is easy to see that Table T(i,j) can be computed in poly(n,k). We first show that the 
“while” loop in Algorithm [2] terminates within at most n steps. In fact, j decreases by 1 each step 
and furthermore j > i > 0 always holds. This is because when j decreases until j = i, j will be 
sampled with probability — = 1; then both j and i will decrease by 1 (Step 

6 — 9). This continues until i = 0. Furthermore, the algorithm terminates with |s| = k because the 
cardinality of s always satisfies |s| = k — i by Step 6 — 8 until the termination at i = 0. Therefore, 
Algorithm [2] runs in poly(n , k) time. 

Now we show that Algorithm 2 outputs s with probability 9 S . Let the output s = {ii,..., ik\ be 
sorted in decreasing order, i.e., i\ > *2 > > ik■ Notice that 

T(i,j) = cnjT[i - 1 , j - 1 ) + T(i,j - 1 ). 
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Figure 1: Comb Sampling 


Therefore, in the Sampling step (Step 5) of Algorithm [2j j is not included to s with probability 
T(i,j — 1 )/T(i,j). Therefore, to sample s = {i\, ...,ik}, it must be the case that n,n — 1, ...,i\ + 1 
are not included, while i\ is included; i\ — 1, + 1 are not included, while i 2 is included; and so 

on so forth. In addition, the sampling in each of these steps is independent and the probability of 
each step is known. Therefore, by multiplying these probabilities together, we have 

_ T(k,n- 1) T(k, n — 2) a^Tjk - 1 ,h ~ 1) 

T(k,n) T(k,n— 1) T(k,i \) 

T(k - Mi - 2) ai k T(0,i k - 1) 

X T(k -Mi - 1)'" T(l,i k ) 

_ fl t<kC^i t 

T(k,n ) 

= e s 

This gives precisely the probability we want. □ 

Remark: we notice that approximately uniform sampling from combinatorial structures has 
been studied in theoretical computer science [7j. Algorithm [ 2 ] uses a variant of the algorithm in |7j, 
and extends their results to the weighted (by 9*) and exact case. 

4.2 Uniform Comb Sampling 

m presented the Comb Sampling algorithm, which randomly samples a pure strategy and achieves 
a given marginal in expectation. The algorithm can be elegantly described as follows (also see 
Figure [I]): thinking of k resources as k buckets with height 1 each, we then put each target, the 
height of which equals precisely its marginal probability, one by one into the buckets. If one bucket 
gets full when filling in a certain target, we move the “rest” of that target to a new empty bucket. 
Continue this until all the targets are filled in, at which time we know that k buckets are also 
full. The algorithm then takes a horizontal line with a uniformly randomly chosen height from 
the interval [0,1], and the k targets intersecting the horizontal line constitute the sampled pure 
strategy. As easily observed, Comb Sampling achieves the marginal coverage in expectation m- 
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However, is Comb Sampling robust against information leakage? We first observe that Comb 
Sampling generates a mixed strategy with support size at most n + 1, which precisely matches the 
upper bound of Caratheodory’s theorem. 

Proposition 1. Comb Sampling generates a mixed strategy which mixes over at most n + 1 
pure strategies. 

Proposition [l] suggests that the mixed strategy sampled by Comb Sampling might be very easy 
to explore. Therefore we propose a variant of the Comb Sampling algorithm. Our key observation 
is that Comb Sampling achieves the marginal coverage regardless of the order of the targets. That 
is, the marginal is still obtained if we randomly shuffle the order of the targets each time before 
sampling, and then fill in them one by one. Therefore, we propose the following Uniform Comb 
Sampling (UniCS) algorithm: 

1. Order the n targets uniformly at random; 

2. fill the targets into the buckets based on the random order, and then apply Comb Sampling. 

Since the order is chosen randomly each time, the mixed strategy implemented by UniCS mixes 
over exponentially many pure strategies, and achieves the marginal. 

Proposition 2. Uniform Comb Sampling (UniCS) achieves the marginal coverage probability. 

5 Experiments 

Traditional algorithms for computing Strong Stackelberg Equilibrium (SSE) only optimize the 
coverage probability at each target, without considering their correlations. In this section, we 
experimentally study how traditional algorithms and our new algorithms perform in presence of 
probabilistic or adversarial information leakage. In particular, we compare the following five algo¬ 
rithms. 

• Traditional: optimal marginal + comb sampling, the traditional way to solve security games 
with no scheduling constraints mm-, 

• OPT: the optimal algorithm for PRIL or ADIL model (Section 3.1) using column generation 
with the defender oracle in Algorithm [lj 

• indepSample: independent sampling without replacement (Section |3.2[ ); 

• MaxEntro: max entropy sampling (Algorithm [2]) ; 

• UniCS: uniform comb sampling (Section 4.2). 

All algorithms are tested on the following two sets of data: 

Los Angeles International Airport (LAX) Checkpoint Data from {12]. This problem 
was modeled as a Bayesian Stackelberg game with multiple adversary types in [1.2]. To be consistent 
with our model, we instead only consider the game against one particular type of adversary - the 
terrorist-type adversary, which is the main concern of the airport. The defender’s rewards and costs 
are obtained from [12] and the game is assumed to be zero-sum in our experiments. 

Simulated Game Payoffs. A systematic examination is conducted with simulated payoffs. 
All generated games have 20 targets and 10 resources. The reward ?’j (cost c*) of each target i is 
chosen uniformly at random from the interval [0,10] ([—10,0]). 
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PRIL in LAX data ADIL in LAX data 



Figure 2: Comparisons on real LAX airport data. 


PRIL With Small Leakage Support PRIL With Full Leakage Support 



PRIL With Uniform Leakage Probability ADIL 



Figure 3: Comparisons in Simulated Games. 
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In terms of running time, all the algorithms run efficiently as expected (terminate within seconds 
using MATLAB) except the optimal algorithm OPT, which takes about 3 minutes per simulated 
game on average. Therefore we mainly compare defender utilities. All the comparisons are listed 
in Figure [ 2 ] (for LAX data) and Figure [3] (for simulated data). The line “Basis” is the utility with 
no leakage and is listed as a basis for utility comparisons. Y-axis is the defender’s utility - the 
higher, the better. We examine the effect of the total probability of leakage (i.e., the x-axis 1 — po) 
on the defender’s utility and consider 1 — po = 0,0.1,..., 1. For probabilistic information leakage, 
we randomly generate the probabilities that each target leaks information with the constraint 
J2?=iPi = 1 — Po- For the case of leakage from small support (for simulated payoffs only), we 
randomly choose a support of size 5. All the utilities are averaged over 50 random games except 
the ADIL model for LAX data. For the simulated payoffs, we also consider a special case of 
uniform leakage probability of each target (see Theorem [ 2 ]). The following observations follow from 
the figures. 

Observation 1. The gap between the line “ Basis ” and “OPT” shows that information leakage 
from even one target does cause dramatic utility decrease to the defender. Moreover, adversarial 
leakage causes more utility loss than probabilistic leakage; leakage from a restricted small support 
of targets causes less utility decrease than from full support. 

Observation 2. The gap between the line “OPP’ and “ Traditional 1 demonstrates the necessity 
of handling information leakage. In particular, the relative loss u(0PT)—u(Basis) is approximately 
half of the relative loss u{Traditional ) — u(Basis) in Figure 3] (and 65% in Figure [ 2 ]). Furthermore, 
if leakage is from a small support (left-up panel in Figure [3), OPT is close to Basis. 

Observation 3. MaxEntro and UniCS have almost the same performance (overlapping in all 
these figures). Both algorithms are almost optimal when the leakage support is the full set [n] (they 
almost overlap with OPT in the right-up and left-down panels in Figure [3]). 

Observation 4. An interesting observation is that IndepSample outperforms Traditional at 
1 — po = 0.3 or 0.4 in all of these figures, which is around ^ ~ 0.37. Furthermore, the gap between 
IndepSample and OPT does not change much at different 1 — pq. 

Observation 5. From a practical view, if the leakage is from a small support, OPT is preferred 
as it admits efficient algorithms (Section |3.1[ ); if the leakage is from a large support, MaxEntropy 
and UniCS are preferred as they can be computed efficiently and are close to optimality. From a 
theoretical perspective, we note that the intriguing performance of IndepSample, MaxEntropy and 
UniCS raises questions for future work. 

6 Conclusions and Discussions 

In this paper, we considered partial information leakage in Stackelberg security games. We focused 
on the one-target leakage case, but do emphasize that our models, hardness results and algorithms 
can be easily generalized. Our results raise several new research questions, e.g., is it possible 
to derive a theoretical approximation guarantee for MaxEntro and UniCS, and can we develop 
efficient algorithms to handle information leakage in other security game settings? More generally, 
it is an interesting problem to study analogous issues of information leakage in other settings beyond 
security, e.g., auctions or general games. 
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7 Appendix A: Derivation of the Defender Oracle 


A defender oracle is a subroutine used to solve security games with large number of pure strategies 
by the Column Generation technique. In this section, we first describe the technique of column 
generation and then derive the formulation for the defender oracle in our models. 

Recall that LP ([2]) has a large number of variables because the number of pure strategies is 
exponential. However, by counting the number of activated constraints at optimality, we know that 
only polynomially many of these pure strategies will have non-zero probabilities at optimality since 
most pure strategies activate the corresponding constraint 0 S > 0 and take probability 0. Column 
generation is based on this observation, i.e., the optimal mixed strategy has a small support. 
Basically, instead of solving LP (J2]) on the set S of all pure strategies, it starts from a small subset 
of pure strategies, denoted as A, and solve the following “restricted” LP. 


maximize p 0 u + J2i=i Pii. u i + v i ) 
subject to u < rjXjj + Cj( 1 — Xjj), 
a, 4 rjXij -(- Cj(xa xj , 

Vi < rj(xjj - Xij ) + Cj(l - Xa - Xjj + Xij ), 
Xij = Y2s£A:i,j£s ® s -> 

Yls&A = 1 

Os >o, 


for j E [ n]. 
for i, j E [n]. 
for i,j E [n]. 
for i,j E [n]. 

for s E A. 


(17) 


Notice that the only difference between LP ([2]) and LP (jT7]) is that the set S of all pure strategies 
is substituted by a small subset A. In practice, A is usually initialized with a small number of pure 
strategies that are arbitrarily chosen. Column generation proceeds roughly as follows: 1. it solves 
LP ( fl~T| ) ; 2. by checking the dual of LP ( fl7| ) the defender oracle judges whether the computed 
optimal solution to LP ( JT7| ) is also optimal to LP ([2]) (setting all pure strategies in S' \ A with 
probability 0); if not, the oracle finds a new pure strategy to be added to the set A and updates 
A. This procedure continues until the defender oracle judges that the computed optimal solution 
w.r.t. current A is also optimal to LP Q. We now explain the underlying rationale of the column 
generation technique. 

We first derive the dual of LP ©■ In fact, to emphasize the key aspects and avoid messy 
derivations, we re-write LP © in the following abstract form: 


maximize d T y 
subject to Mx + Ny < c 

X H ~ J2s£A-.i,jes °s = 0, for i,j E [n], (18) 

X)seA 0s = 1 

0 S > 0, for s E A. 


where variable y represents the vector consisting of u,Vi,Ui while variable x is the vector repre¬ 
sentation of x^ (putting i,j in some fixed order); d is a vector summarizing the original objective 
coefficients; the constraints Mx + Ny < c summarizes the first three set of constraints in LP ( |17[ ). 
This abstract form not only simplifies our derivation of the dual, more importantly it emphasizes 
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that the column generation technique works regardless of what the first three sets of constraints 
are as long as there are polynomially many of them. 

Let M index u^ be the column vector of M corresponding to variable Xjj and Ni be the column 
vector of N corresponding to the Z’th component of y. We can now simply derive the dual of 


LP (18) as follows: 


minimize 
subject to 


c T p +10 


p r Ni > d h 

P Mind,ex(i,j) T Pij>0, 
~ Ei.es Pij + W > 0, 

p > 0 


for all l. 
for i,j E [n] 
for s E A. 


(19) 


where p are the dual variables w.r.t. the first set of constraints in LP (18) and (3ij, co are the dual 
variables w.r.t. the second and third set of constraints. 


First notice that the optimal solution to LP (18) (denoted as OptSolA ) and the optimal solution 


to LP (19) (denoted as OptSolDualA ) can both be computed efficiently when A is small. A key 
observation here is that, if OptSolDualA, in particular, ui and happens to make the constraints 
— Eijes Pij + w > 0, Vs E A hold more generally as — Eijes Ai + w > 0, Vs E S, then we claim 
that the OptSolA is also an optimal solution to LP ([2]) (by picking pure strategies in S \ A with 
probability 0). This is because, if we substitute A by S in both LP (18) and LP (19), OptSolA 


is still feasible to LP (18) because all the newly added strategies (in S \ A) have probability 0; 
OptSolDualA is still feasible to LP (19) because our uj,f3ij make constraints —EijesPij + ^ > 0 
hold for all s E S by assumption. Furthermore, complementary slackness still holds since the added 


new variables in LP (18) all take value 0. By linear program basics, we know that OptSolA is still 


optimal if we substitute A in LP (18) by S, which is precisely LP ([2]). 

As a result, our key task is to judge whether — - es f3ij + u > 0 holds for all s E S for a given 


dual solution. This is equivalent to decide whether to > max sg g 
is then defined as the following problem: 


Elijes Aj 


The defender oracle 


max 

seS 


i,jes 


v 


= maxs' ( 
seS 


T M + M t , 


( 20 ) 


where M is the matrix satisfying Mjj = . In other words, the defender oracle finds a pure 

strategy s that maximizes the sum Eijes^r 

With this oracle, column generation proceeds, in more details, as follows: 1. compute LP (18) 


and LP (19); 2. use the defender oracle to solve Problem (20): if the optimal value is less than 
or equal to the dual variable io, asserts optimality; otherwise, add s* - the optimal solution to 
Problem (20) - to A; 3. repeat until optimality is reached. Notice that the newly added s* does 
not belong to the original A because all s E A satisfy Eijes Aj < Column generation does not 
guarantee polynomial convergence, but usually converges very fast in practice. This is because the 
optimal mixed strategy usually has a small support. 


When information leaks from a small subset of targets (Section 3.1), the set of variables x. 




become smaller since only correlation between leaking targets and other targets are considered. By 


modifying LP (18) and LP (19) a bit, we can get the defender oracle formulation as described in 
Section 13.11 
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8 Appendix B: Relation between V(n, k ) and the Correlation Poly¬ 
tope 

In this section, we show a connection between V(n,k) and the correlation polytope, defined as 
follows: 

Definition 1. JT5 j / Given an integer n, the Correlation Polytope V(n) is defined as follows 

V{n) = Conv ({ vv T : v G {0, l} n }) . 

where Conv(S) denotes the convex hull of set S. Notice that vv T £ {0, l} nxn . 

The following lemma captures the relation between V(n) and V(n, k). 

Lemma 8. X £ V(n, k) if and only if the following three constraints hold: (a) X £ V(n); (b) 
tr ( x ) = E?=i *« = k ; and (c) sum(X) = Y7ij=i x ij = • I n other words, V(n,k) is decided by 

V(n) with two additional linear constraints. 

Proof. We show that, given X £ V(n), if X satisfies the following two linear constraints: tr(X ) = 
k, sum(X) = k 2 , then X £ V(n, k). 

Since X £ V(n), there exits X{ G V(n , z) and pi > 0, such that X = P% x i an d EILi Vi = 1- 
That is, X is a convex combination of elements from each V(n,i). Notice that VXj G V{n,i ), we 
have tr{Xi) = z and sum^Xf) = z 2 , since any vertex of V(n,i) satisfies these constraints. Let 
X G V(n, k), then we have: 


n 

(z) :1 = ^Pi 

i =1 

n n 

(zz) :k = tr(X) = ^PiX tr(Xi) = s ^p i xi 

i= 1 Z—1 

n n 

(zzz) :/z 2 = sum(X) = ^^pi x sum^Xf) = x z 2 

Z=1 Z=1 

By the Cauchy-Schwarz inequality, we have P*)(Ez=i VP 2 ) ^ 027=1 PP) 2 ■ Plugging in 

the above three equations into the Cauchy-Schwarz inequality yields that the equality holds. The 
condition of equality for the Cauchy-Schwarz inequality is that pfi 2 /pi is a constant for all z, such 
that pi 0. This shows that there is only one non-zero among pf s. That is p^ = 1. Therefore, 
XeV(n,k). □ 

Remark: we note that m defines correlation polytope in a more general fashion and our 
definition of V(n) is in fact an important special case of the definition of correlation polytope in 
m which is called the full correlation polytope. Nevertheless, this definition is sufficient for our 
model. m proved that membership check for polytope V(n) is NP-complete. Lemma [8] basically 
conveys that optimizing over polytope V(n, k ) is no harder than optimizing over V(n). Nevertheless, 
Lemma [4] shows that optimizing over V(n,k) is still NP-hard. 
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